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ABSTRACT 

This  study  seeks  to  examine  the  verity  of  the  common  perception  that  single-bid  orders 
are  more  costly,  lower  in  performance  quality,  and  shorter  in  pre-award  process  time 
than  competitively-bid  orders.  To  this  end,  we  look  at  why  single-bid  orders  exist  in  the 
current  procurement  system,  how  these  orders  measure  up  to  competitively-bid  orders  in 
cost  and  performance,  and  how  customer  satisfaction  varies  over  time  and  with  the 
number  of  bids  received.  60  orders  from  six  different  government  agencies  under  the 
GSA  Service  Schedules  were  analyzed.  Contract  files  were  reviewed,  interviews  with 
both  contract  and  program  officers  were  conducted,  and  the  resultant  data  were 
examined  using  a  variety  of  quantitative  analysis  methods. 
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ABSTRACT 

This  study  seeks  to  examine  the  verity  of  the  common  perception  that  single-bid  orders 
are  more  costly,  lower  in  performance  quality,  and  shorter  in  pre-award  process  time 
than  competitively-bid  orders.  To  this  end,  we  look  at  why  single-bid  orders  exist  in  the 
current  procurement  system,  how  these  orders  measure  up  to  competitively-bid  orders  in 
cost  and  performance,  and  how  customer  satisfaction  varies  over  time  and  with  the 
number  of  bids  received.  60  orders  from  six  different  government  agencies  under  the 
GSA  Service  Schedules  were  analyzed.  Contract  files  were  reviewed,  interviews  with 
both  contract  and  program  officers  were  conducted,  and  the  resultant  data  were 
examined  using  a  variety  of  quantitative  analysis  methods. 


EXECUTIVE  SUMMARY 

In  examining  how  satisfaction  and  cost  vary  across  single-bid  and  multi-bid  contracts  and 
how  government  satisfaction  has  changed  over  time,  we  obtained  the  following  results.  In 
short,  we  found  that  single-bid  orders  are  not  more  costly,  are  not  lower  in  performance 
quality,  and  that  award  process  time  is  not  related  to  the  number  of  bids.  More  detailed 
outcomes  are  available  in  the  Analysis  and  Discussion  sections. 

SATISFACTION 
High  Satisfaction  Levels 

Our  findings  reveal  that  the  current  process  for  selecting  vendors  results  in  high  levels  of 
customer  satisfaction.  We  know  from  contract  reviews  that  a  standard  approach  to  “best 
value”  is  judged  on  the  breadth  of  technical  expertise,  the  extent  of  past  performance  or 
corporate  excellence,  personnel  qualification,  and  price;  and  while  the  weights  assigned 
to  each  criterion  vary  by  agency,  technical  expertise  and  past  performance  usually  rank 
above  price.  The  customer  satisfaction  measures  we  relied  on  address  satisfaction  with 
this  best  value  judgment.  Overall,  we  observed  low  dissatisfaction  levels  and  very  high 
ratings  on  vendor  accomplishment,  delivery,  cooperation  and  prices. 


Improved  Satisfaction  Over  Time 

Analysis  shows  satisfaction  improved  over  time.  Compared  to  a  similar  study  done 
fourteen  years  ago  by  Dr.  Steve  Kelman,  there  are  marked  improvements  in  satisfaction 
within  the  procurement  process.  Acknowledging  for  differences  between  this  study  and 
that  of  Kelman,  vendor  performance  and  customer  satisfaction  improved  substantially  on 
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identical  questions  between  the  two  studies.  This  change  is  likely  due  to  a  combination 
of  new  legislation  in  the  1990s,  greater  customer  responsibility,  and  a  greater  demand  for 
quality  services. 


No  Decrease  in  Satisfaction  in  Single-bid  v.  Multi-bid 

Satisfaction  did  not  decrease  with  the  number  of  bids  on  a  given  contract;  in  fact, 
single-bid  and  sole  source  awards  had  lower  dissatisfaction  ratings  than  multi-bid  awards. 
Similarly,  they  have  higher  satisfaction  ratings  in  all  other  categories.  These  observations 
do  not  mean  we  commit  to  the  extreme  and  say  all  single-bids  are  great  for  government 
business;  rather,  we  are  hesitant  to  condemn  single-bid  awards  under  current  procurement 
regulations. 

Recommendation:  Maintain  existing  legislative  reforms.  Clearly  it  has  been  effective 
thus  far.  The  role  of  Contract  Officers  has  changed  over  time,  as  the  FAR  allows  them 
leeway  to  operate  on  their  best  business  judgment.  These  reforms  have  improved 
Contract  Officer  discretion  in  the  process  and  ultimately  led  to  an  increase  in  satisfaction 
with  the  process.  More  streamlining  of  the  FAR  to  make  information  more  easily 
accessible  could  be  warranted. 


PRICE  AND  PROCESS  LENGTH 

No  Significance  Between  Single-bid  and  Contract  Value 

There  is  no  statistical  relationship  between  single-bids  and  contract  value.  Contrary 
to  expectations,  analysis  did  not  support  that  single  bids  are  higher  in  value  (with 
significance).  In  a  comparison  of  means,  multi-bid  and  single-bid  awards  have  similar 
average  values;  however,  sole  source  awards  are  higher  in  mean  value.  . 

Analysis  also  shows  that  the  length  of  time  to  get  under  contract  remains  an  issue  for  sole 
source  awards.  For  example,  we  know  that  one  third  of  the  sole  source  awards  reviewed 
had  some  relation  to  time  constraints;  and  in  five  cases  customers  failed  to  allocate  more 
than  four  days  for  the  awards  process.  Compared  to  multi-bid  awards  that  averaged  6.6 
weeks,  sole  source  awards  averaged  2.8  weeks  in  award  process  time. 

Findings  show  that  increases  in  award  process  time  are  associated  with  increases  in  all 
satisfaction  levels,  with  the  exception  of  vendor  cooperation.  This  is  likely  due  to  more 
opportunities  for  negotiating  a  fair  price,  more  time  to  specify  the  vendor  role,  more  time 
evaluating  best  value,  and  more  time  competing  the  project. 
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No  Significance  in  Discounts 

We  speculated  that  lower  competition  would  be  associated  with  lower  discounts  or  no 
discoimts;  however,  there  is  no  statistical  significance  to  support  this  claim.  This  is 
likely  the  result  of  a  difficulty  in  obtaining  reliable  data.  Furthermore,  many 
immeasurable  factors  influence  discounts.  Discounts  are  usually  offered  for  three 
reasons:  Vendors  want  to  continue  work  on  future  government  contracts;  vendors  have  a 
history  of  good  relationships  with  the  customer;  and  vendors  think  others  are  competing 
for  the  award. 


Relationship  Between  Increases  in  Time  and  Contract  Value 

We  determine  that  the  length  of  award  process  time  is  associated  with  contract  value.  An 
increase  in  the  length  of  the  award  process  is  associated  with  an  increase  in  contract 
value.  We  do  not,  however,  assert  diat  time  increases  the  contract  value;  rather,  larger 
contracts  may  necessitate  a  lengthening  of  the  award  process  to  allow  for  competition.  In 
fact,  we  find  that  efforts  to  compete  are  associated  with  an  increase  in  contract  value  of 
between  57  and  69  percent  relative  to  no  efforts  to  compete  -  the  higher  the  value,  the 
more  likely  the  search  for  competition. 

Recommendation:  Customers  engaged  in  advanced  planning  would  likely  eliminate 
the  higher  costs  associated  with  single  bids.  Improvement  may  be  needed  in  this  area. 
The  reluctance  to  compete  contracts  based  on  a  need  for  quick  turnaround  time  on 
contract  renewals  is  a  poor  excuse  for  urgency. 

We  advise  customers  to  engage  in  forward  planning.  Some  vendors  are  known  to 
operate  company-specific  information  technology  systems  that  would  require  future  sole 
source  follow-on  contracts  for  additional  project  work,  network  maintenance,  or 
continued  database  management.  Selecting  a  vendor  known  to  have  systems  associated 
with  high  transition  costs  for  contract  renewal  is  poor  management  practice. 


HIDDEN  COMPETITION 

Hidden  Competition  Affects  The  Number  of  Bids 

The  evidence  suggests  that  ‘‘hidden  competition”  (where  vendors  are  intimidated  by  the 
presence  of  one  or  more  vendors  they  perceive  to  have  an  advantage,  and  as  a  result  fail 
to  bid)  does  play  a  role  in  how  many  vendors  bid  on  an  award.  While  this  provokes 
the  obvious  recommendation  calling  for  increasing  efforts  to  ensure  vendors  are  not 
dissuaded  fi-om  competing,  the  small  number  of  “hidden  competition”  awards  (10  of  60) 
precludes  a  clear  judgment  to  this  effect. 


3 


Cj  S  A 


U.S.  General  Services  Adminisbration 


Recommendation:  Attention  to  hidden  competition  is  warranted.  Despite  our  hesitance 
to  draw  a  specific  conclusion,  logic  dictates  that  fair  competition  is  paramount. 
Responsible  agencies  will  try  to  compete  large  businesses  with  others,  small  businesses 
with  others,  or  if  mixed,  give  small  businesses  some  advantage  in  the  bidding  process  to 
ensure  fair  competition. 


STATEMENT  OF  WORK 

Relationship  Between  Statement  of  Work  and  Satisfaction 

An  increase  in  the  Statement  of  Work  (SOW)  quality  rating  is  associated  with  an 
increase  in  customer  satisfaction.  This  is  not  a  surprising  trend,  given  that  several 
Contract  Officers  complained  of  problems  with  SOW  quality  during  the  award  process. 
Sloppy  or  vague  SOWs  leave  the  vendor  with  an  incomplete  or  incoherent  set  of 
guidelines  for  the  contracted  work.  As  such,  SOWs  of  lesser  quality  would  increase 
prices  and  decrease  work  quality.  Our  data  supports  this  trend.  In  fact,  increasing  the 
quality  rating  of  SOWs  by  one  point  on  a  scale  of  one  to  five  increases  the  three 
satisfaction  ratings  by  .4  points.  With  already  high  average  ratings  above  9.4,  this  small 
change  would  result  in  a  relatively  large  increase  in  satisfaction. 

Recommendation:  Improving  the  SOW  increases  customer  satisfaction  on 
performance.  Specific  SOWs  define  the  vendor  role,  while  vague  SOWs  can  leave  much 
up  to  vendor  discretion  and  lead  to  cost  overruns.  Given  the  nature  of  best  value 
judgments,  it  is  in  the  best  interest  of  each  customer  to  improve  or  keep  high  standards 
for  the  SOW.  This  calls  for  well- written  and  detailed  SOWs,  which  would  also  preclude 
unnecessary  increases  in  award  process  time. 
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INTRODUCTION 

In  the  century  the  role  of  government  will  shift  fi-om  its  more  historic  mission  of 
being  primarily  the  provider  of  goods  and  services  to  the  role  of  being  the  manager  of  the 
providers  of  the  goods  and  services.  The  government  is  changing  fi'om  hiring  people 
who  are  more  of  the  “doers”  to  hiring  people  with  the  skills  to  manage  and  oversee  the 
“doers” —  the  latter  largely  coming  firom  the  competitive  private  sector* . 

This  more  involved  public-private  sector  partnership  within  the  last  decade  is  the  new 
way  of  conducting  federal  procurement  and  acquisitions.  Reform  in  the  past  decade 
brought  contracting  for  goods  and  services  to  new  heights,  as  reduced  costs  to  federal 
agencies  and  taxpayers  engendered  bureaucratic  reform,  and  increased  both  large  and 
small  business  awards.  Thus,  the  acquisition  reform  results  seem  to  be  demonstrating 
that  not  only  does  the  government  get  better  products  and  services  at  lower  prices  fi'om 
contracting,  but  that  the  supplier  community  is  better  off  and  more  competitive.  There 
are  many  opporhmities  to  assure  that  reforms  are  continued  into  the  future,  and  that 
others  are  introduced  to  maintain  the  momentum.  Nevertheless,  in  spite  of  the  initial 
success  of  the  acquisition  reform  efforts  during  the  1990s,  problems  still  remain  within 
the  acquisition  process. 

Conventional  wisdom  tells  us  that  one  of  these  problems  is  single-bid  orders.  It  is  a 
common  perception  that  single-bid  contracts  are  more  costly,  lower  in  performance 
quality,  and  shorter  in  pre-award  process  time  than  competitively-bid  orders.  We  address 
this  concern  by  examining  contracts  off  the  General  Services  Administration  (GSA) 
Federal  Supply  Schedules  (FSS)  for  information  technology  (IT)  contracts.  Specifically, 
there  is  fear  that  federal  agencies  (GSA  customers)  are  awarding  a  large  number  of  IT 
service  contracts  without  adequate  competition  from  private  sector  firms  (vendors).  As 
such,  government  agencies  might  stand  to  lose  from  lack  of  adequate  competition; 
bypassing  any  competition  during  the  bidding  phase  could  amount  to  excess  costs  and 
decreased  quality  for  outsourced  IT  services.  Much  worse  is  the  possibility  of  corruption, 
favoritism,  laziness,  sloppiness  or  haste  from  contracting  officers  in  the  award  process.  If 
such  a  trend  exists,  the  effectiveness  of  federal  procurement  policy  and  the  mandated 
processes  by  which  contracting  agencies  must  follow  become  questionable,  and  would 
require  lawmakers  to  adjust  a  decade’s  worth  of  legislation  to  improve  the  federal 
procurement  process. 


*  The  Procurement  Revolution.  Ed.  Abramson,  Mark  A.  and  Roland  S.  Harris  III.  New  York:  Rowman 
and  Littlefield  Publishers  Inc,  2003.  p.  16 
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With  this  study,  we  analyze  and  discuss  the  following: 

•  How  satisfaction  and  cost  varies  between  single-bid  and  multi-bid 
contracts 

•  How  government  satisfaction  with  the  procurement  process  has  changed 
over  time 

We  will  discuss  whether  there  are  a  large  number  of  single-bid  contracts,  what  causes 
single-bid  contracts,  how  the  procurement  process  and  vendor  performance  has  changed, 
and  if  there  is  a  need  for  a  better  way  to  handle  single-bid  contracts  off  the  GSA  Service 
Schedules. 
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BACKGROUND 

The  U.S.  General  Services  Administration  (GSA) 

The  client,  GSA,  enters  into  contracts  with  commercial  firms  to  provide  supplies  and 
services  at  specified  prices  for  given  periods  of  time.  Their  programs  mirror  commercial 
buying  practices  more  than  any  other  procurement  process  in  the  federal  government. 
They  provide  customers  with  millions  of  state-of-the-art,  high-quality  commercial 
products  and  related  services  at  lower  prices  than  if  agencies  were  to  directly  procure 
goods  and  services  on  their  own. 

Fimded  by  the  federal  government,  GSA  is  comprised  of  13,000  associates  who  support 
over  one  million  federal  workers.  The  role  of  GSA  is  to  aid  federal  agencies  in  buying 
space,  products  and  services.  They  do  this  by  contracting  with  federal  and  commercial 
sources.  By  leveraging  the  volume  of  the  federal  market  and  negotiating  large,  multi¬ 
user  contracts,  GSA  is  able  to  drive  down  prices  for  federal  agencies.  For  businesses,  the 
advantage  is  the  opportunity  to  sell  billions  of  dollars  worth  of  products  and  services. 

The  Federal  Acquisition  Regulation 

IT  service  contracts  using  Federal  Supply  Schedules  fall  under  the  Federal  Acquisition 
Regulation  (FAR).  This  statutory  provision  has  institutionalized  the  preference  for  the 
use  of  commercial  products  and  commercial  practices  in  government  contracting. 

GSA  awards  and  administers  Federal  Supply  Schedules  through  authority  fi-om  the 
Federal  Property  Administrative  Services  Act  of  1949.  Pursuant  to  this  legislation,  GSA 
enters  into  government-wide  contracts  with  private  vendors  to  provide  a  number  of 
products  and  services  for  given  periods  of  time. 

GSA  determines  contracts  ordered  off  of  the  Federal  Supply  Schedules  are  priced  as  “fair 
and  reasonable”.^  The  FAR  mandates  that  Request  for  Quotations  (RFQs)  be  sent  to  at 
least  three  vendors  to  attract  more  than  one  bid  in  the  award  process.  While  there  is  no 
statutory  requirement  on  number  of  bids  for  government  agencies  using  the  Federal 
Supply  Schedules,  the  number  of  single-bid  contract  awards  (awards  where  only  one  bid 
is  received,  though  more  than  one  vendor  was  solicited)  is  the  focus  of  this  study.  It 
suggests  concern  with  the  competitive  processes  stipulated  in  the  FAR.  Ideally, 
competition  results  in  the  best  value  for  the  government;  the  lack  of  competition  in 
single-bid  awards  may  have  a  number  of  adverse  implications  to  the  value  of  contracted 
services. 


^  [FAR  8.4] 
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The  existence  of  single-bids  also  suggests  the  traditional  method  of  sole  source  (awards 
where  no  competition  is  deemed  necessary)  is  avoided,  or  being  used  sparingly  as 
increased  scrutiny  may  be  a  disincentive  for  using  the  sole  somce  provision.  Under  the 
FAR,  customers  must  not  only  clearly  specify  all  the  services  available  within  their 
request,  but  also  ensure  that  each  contractor  is  given  a  fair  opportunity  to  be  considered; 
and  while  FAR  does  grant  exceptions  to  the  rule,  such  awards  are  intended  to  be  used  in 
moderation.  Part  6.3  of  the  FAR  stipulates  that  exceptions  are  warranted.  These  include 
situations  where  only  one  responsible  somce — and  no  other  supplies  or  services — ^will 
satisfy  agency  requirements,  as  well  as  events  or  work  with  imusual  and  compelling 
mgency. 

Procmement  approach,  procedme  and  cultme  are  some  of  the  additional  changes  beyond 
the  statutory  requirements.  Prior  to  procmement  reform,  there  was  very  little  actual  fraud 
or  other  illegal  actions,  but  there  was  enormous  waste  in  government  procmement.  The 
problem  was  not  the  people,  but  the  processes  being  used.  With  a  wealth  of  legislation 
passed  since  the  days  of  excess  government  waste,  that  problem  has  been  remedied.  Yet, 
broader  issues  of  erroneous  contract  approvals  have  now  emerged  with  a  world  of 
increased  government  outsomcing — ^particularly  in  IT  services — ^hinting  that  if  this  is  a 
problem,  it  might  rest  with  the  people  managing  the  bidding  process. 

Literatme  reveals  little  on  the  problem  of  contracting  officer  subjectivity  in  the  bidding 
process,  thus  suggesting  data  gaps  in  the  area  of  intent;  however,  it  is  known  that  the 
extent  of  discretion  by  contracting  officers  has  evolved  over  time.  Previously, 
contracting  officers  assessing  what  action  they  could  or  could  not  take  on  a  particular 
case  would  search  regulations  and  procedmal  manuals  to  find  a  specific  authorization. 
Now  contracting  officers  structme  solutions  based  on  their  best  business  judgment  and 
search  to  see  if  there  is  a  prohibition.  This  change  in  authority  is  articulated  in  the  FAR, 
stating  that  members  of  the  procmement  team  may  use  their  best  judgment  if  a  specific 
strategy,  practice,  policy  or  procedure  is  in  the  best  interests  of  the  government.^ 


^  [FAR  1.102(d)] 
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RESEARCH  DESIGN 

This  study  had  the  following  major  design  features: 

•  Contract  files  reviewed  were  IT  service  contracts  from  the  GSA  Service 
Schedules. 

•  Contracts  comprised  a  value  range  of  $500,000  and  above.  Dollar  values 
were  obtained  from  the  Federal  Procurement  Data  System  (FPDS)  and 
contract  files. 

•  Contracts  were  awarded  between  January  2002  and  June  2003. 

•  60  orders  were  reviewed,  and  79  interviews  conducted.  All  contract 
appraisals  and  in-person  interviews  took  place  at  government  agencies  within 
the  Washington,  DC  area.  Contract  and  Program  Officers  were  interviewed 
both  face-to-face  and  via  telephone. 


The  Data 

We  collected  three  types 
of  data  over  the  course  of 
three  weeks  for 
subsequent  analysis: 
Contract  data,  customer 
satisfaction  data  and 
open-ended  response 
data.  A  full  list  of  data 
obtained  from  both 
interviews  and  contract 
research  is  documented  in 
the  information  boxes. 

Contract  data  (Box  1) 
represents  values  and 
information  directly 
pulled  from  contract  files, 
and  includes  interviews 
with  Contract  Officers 
(KOs)  to  obtain  data  not 
readily  apparent  in  the 
files.  This  data  consists 
of  variables  such  as 
contract  value,  discounts. 


. : . I . _ . .i '..w 


f*'-/ 1.-, 

i‘  -  • 


‘  ‘9.  If  fixed  price,  was  there  a  not  To  exceed  Amount  in  the 


TO,  How  was  the  price  determined?'  Was  there;, negotiation?;  . 
1  i;  Is  the  customer  satisfied  with  the  contractor?^,'  U'.if,  V  ■  ,  --  ■ 
|I®AreifhBfeTjenortrmnee;metnc5&is#p|ia®8» 

■  '-'.I  <i  ,  .  >  •  .  ft..,!*  4' 

->»/•  ini  t  ex's 


16.  Was  the  discount  negotiated?  '  ' 

17.  Upon  order  completion,  was  the  final  price  different  than 
the  fixed  price?  If  so,  how  much?  .  >  i,  ■ 


19.  What  is  the  Contract  value? 
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competition,  and  length  of  award  process.  “Yes/no”  answers  were  coded  as  dummy 
variables,  while  actual  values  were  used  for  all  quantifiable  variables. 


Customer  satisfaction  data  (Box  2)  represents  information  obtained  from  in-person 
interviews.  Both  Contract  and  Program  Officers  were  interviewed.  Contract  Officer 


H^On  lhis‘'contract;  what  perccn'tagem5f  time ■Wr^Sfjrou'- dissatisfied  with  the  pLiloimai'LC  ihis 
P'"  \endor(onaO  100%scale)‘^  •'  .  ■  - 


ao  10,  rate>'cndpr;p£^fprrnance{mdCT(the;,ofdCT,.onlhc  following  ilinien'.ions 
:yendoraccomplished-what we.;^,w^ted.themto.,'I,  'i-*  s- y ^ 


•  •f«4Theyendor  accomplished-’^ 


ice  was-  fat, 


interviews  were  conducted  at  the  agency  site  with  the  contract  in-hand  to  familiarize  the 
interviewee  with  the  vendor.  Program  Officers  (POs)  were  interviewed  via  telephone. 
Because  they  worked  with  the  vendor  on  a  daily  basis,  it  was  not  necessary  to  be  present 
with  the  contract  in  hand. 

Also,  customer  offices  were  located  outside  our  travel  proximity,  which  precluded  in- 
person  interviews.  Questions  1,  2  and  4  were  taken  from  Dr.  Steve  Kehnan’s  1990  book. 
Procurement  and  Public  Management,  in  which  he  conducted  a  study  of  vendor 
performance  and  government  satisfaction  prior  to  procurement  reform.  Part  of  this  study 
looks  at  customer  satisfaction  over  time  and  across  different  types  of  contracts,  using 
Kelman’s  results  as  a  point  of  comparison. 

Open-ended  response  data  represents  qualitative  responses  to  voluntary  follow-up 
questions.  The  questions  listed  in  Box  3  were  sent  to  Contract  Officers  over  email  after 
in-person  interviews  were  conducted.  We  used  this  information  to  understand  the 
procurement  process  from  the  individual  participant,  gain  insight  on  organizational 
culture  and  determine  revealing  trends  or  patterns  amongst  Contract  Officer  responses. 
These  questions  are  useful  firom  a  management  perspective,  but  bear  little  relevance  to 
the  data  analysis  other  than  background  information.  The  answers  collected  from  these  e- 
mail  interviews  will  buttress  our  findings  from  regression  and  fi-equency  distribution  data. 
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Randomization 

The  exact  criteria  for  obtaining  a  sample  size  were: 

1)  all  orders  must  be  IT  service  contracts  off  the  GSA  Service  Schedules, 

2)  all  orders  must  be  greater  than  $500,000  in  value,  and 

3)  all  orders  must  be  awarded  between  6  and  18  months  prior  to  January  1,  2004. 

While  the  search  returned  a  number  of  agencies  across  the  United  States — all  adequate 
for  this  study — ^we  were  constrained  to  the  Washington,  DC  area  for  travel,  schedule  and 
financial  considerations.  GSA  selected  government  agencies  with  the  most  orders 
returned  through  the  FPDS  search,  and  those  agencies  that  agreed  to  participate  in  this 
study  were  chosen. 

As  such,  there  may  be  selection  bias  in  our  results.  The  sample  does  not  fully  represent 
the  whole  population  of  government  agencies  because  our  study  focused  only  on  agencies 
that  agreed  to  participate.  It  follows  that  agencies  with  nothing  amiss  would  be  happy  to 
participate  in  our  study,  whereas  an  agency  with  something  to  hide  would  not.  For  this 
reason,  it  could  be  the  case  that  we  saw  only  agencies  that  were  well-organized,  had 
competent  KOs,  and  worked  well  with  the  POs. 

We  visited  six  government  agencies  in  the  DC  area.  There  is  a  possibility  that  some  third 
variable  is  associated  with  being  based  in  DC;  perhaps,  for  whatever  reason,  agencies 
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based  there  are  more  efficient,  more  fair,  less  efficient,  or  less  fair  because  of  their 
location.  With  more  agencies  from  different  locations  in  the  sample,  our  selection  would 
have  been  more  representative.  Further,  each  agency  had  a  large  enough  share  of  the 
total  contracts  to  skew  the  results  (had  any  agency  had  significantly  different  results  than 
the  others — ^they  did  not). 

Oiu  sample  is  representative  of  a  broad  spectrum  of  government  sectors.  We  believe  this 
will  counter  some  of  this  study’s  limitations. 


Other  Limitations  and  Biases 
Access 

The  remaining  limitations  came  from  limits  in  access  to  Contract  and  Program  Officers. 
This  explains  the  disparity  in  number  of  Contract  versus  Program  Officers  interviewed. 
Some  original  Contract  Officers  were  no  longer  working  for  the  agency  or  had  different 
contact  information.  In  these  cases  we  were  forced  to  interview  the  presiding  Contract 
Officer  or  manager.  These  individuals  did  not  originally  work  with  the  client,  and  had 
trouble  relating  to  the  customer  satisfaction  questions.  This  might  also  bias  the  results  if 
those  interviewing  on  behalf  of  an  old  employee  would  negatively  comment  on  the 
contract  because  it  was  not  their  own  work;  however,  the  virtually  uniform  positive 
answers  alleviate  much  of  this  concern. 

Program  Officers  were  not  located  at  the  agency  site,  and  we  were  limited  to  phone 
interviews.  Only  a  small  sample  of  POs  were  contacted  for  much  of  the  same  reasons 
stated  above.  Interviewing  POs  was  useful  in  obtaining  information  fi-om  a  different 
perspective;  however  the  small  number  of  POs  interviewed  will  limit  our  findings. 

The  Contract  Officer  provided  contact  information  for  Program  Officer  interviews.  This 
is  another  indication  of  selection  bias.  KOs  could  withhold  information  on  POs  with  bad 
experiences  or  who  docmnented  poor  performing  vendors.  Once  we  contacted  Program 
Officers,  they  had  a  choice  of  whether  or  not  to  conduct  the  interview,  which  could  also 
be  a  source  of  bias.  However,  the  POs  decided  whether  they  would  participate  in  the 
interview  before  they  knew  which  order  we  would  be  asking  them  about,  so  it  is  probable 
that  this  bias — ^if  it  does  exist — ^is  negligible. 
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Distorting  Truth 

We  have  no  reason  to  believe  that  either  Contract  or  Program  Officers  would  distort  the 
truth.  Each  interview  contained  a  review  of  the  Non-Disclosure  Agreement,  which  was 
signed  and  circulated  throughout  all  agencies  visited.  This  eliminated  the  incentive  to  lie 
about  the  order  in  any  way.  While  one  agency  did  significantly  limit  the  information  they 
allowed  Contract  Officers  to  share  by  “scrubbing”  each  file  prior  to  our  arrival,  the 
objective  was  to  limit  what  proprietary  information  could  be  safely  shared  with  Harvard 
University  researchers,  and  not  to  distort  the  truth. 

Sequencing  questions  did  not  affect  the  truthfulness  of  answers.  The  majority  of  Contract 
Officers  commented  on  more  than  one  contract,  experiencing  the  same  set  of  questions 
more  than  once.  Some  gave  vendors  positive  ratings  at  the  beginning  of  the  interview, 
and  rated  poor  performing  vendors  harshly  in  subsequent  questioning.  If  they  were 
skewing  their  answers,  they  would  have  had  ample  time  to  prepare  for  and  mask  their 
responses.  This,  however,  might  not  be  the  case  for  Program  Officers.  On  average. 
Program  Officers  were  questioned  on  one  vendor,  and  these  interviewees  were  not 
familiar  with  the  questioning.  While  we  did  inform  them  of  the  Non-Disclosure 
Agreement,  it  is  possible  we  have  some  truth  distortion  in  this  smaller  interview  sample. 
Truth  distortion,  though,  would  amount  to  only  a  small  bias,  if  any.  As  we  will  see,  there 
is  little  variance  between  Contract  Officer  and  Program  Officer  responses  to  the  same 
questions  regarding  the  same  vendors  and  contracts. 
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ANALYSIS 

Recall  that  our  objectives  in  this  study  are  to  determine  the  following: 

•  How  satisfaction  and  cost  vary  across  single-bid  and  multi-bid  contracts 

•  How  government  satisfaction  has  changed  over  time 

In  beginning  this  study,  we  speculated  that  single-bid  contracts  would  be  more  costly, 
lower  in  performance  quality,  and  shorter  in  pre-award  process  time.  Indeed,  much  of  the 
analysis  is  focused  on  examining  this  statement;  however,  we  also  explore  the  alternative: 
If  single-bid  contracts  are  no  different  than  multi-bid  contracts,  what  else  drives  variables 
such  as  contract  value,  discounts,  and  performance?  We  find  significant  information  by 
reaching  beyond  the  single-  and  multi-bid  variables  to  understand  the  factors  driving  best 
value  in  government  procurement. 

Summary  Statistics  of  Contract  and  Program  Officers 

Customer  satisfaction  data  is  separately  comprised  of  Contract  and  Program  Officer 
responses.  We  combine  these  two  populations  in  later  satisfaction  analysis  to  increase 
the  sample  size  of  customer  satisfaction  data  and  associated  significance. 

Table  1  offers  a  glimpse  of  the  satisfaction  ratings  across  contract  and  program  officers, 
and  states  the  case  for  the  similarity  between  Contract  and  Program  Officers.  In  all  cases, 
the  majority  of  respondents  gave  scores  of  10.  An  even  higher  percentage  gave  scores  of 
9  or  10.  For  example,  among  Contract  and  Program  Officers  combined,  67.1  percent 
gave  the  vendor  in  question  a  10  and  84.9  percent  gave  the  vendor  in  question  a  9  or  a  10 
on  their  performance  on  keeping  promises. 


Table  1 


Percent  of  Combined  Contract  and  Program  Officer  Satisfaction  Rating 


Rating 

Promises 

Delivery 

Accomplishment 

Cooperation 

Fair  Price 

10 

67.1 

62.7 

78.4 

76.3 

79.2 

9  or  10 

84.9 

86.7 

93.2 

88.2 

92.2 

The  average  percent  time  dissatisfied  with  the  vendor  among  Contract  Officers  for  the 
contract  in  question  is  2.04  percent  overall.  This  is  reduced  to  1.33  percent  if  we  exclude 
the  outlier  of  40  percent.  However,  88  percent  of  KOs  were  not  dissatisfied  at  all. 
Among  the  12  percent  who  were  dissatisfied,  the  percentage  of  time  they  were 
dissatisfied  ranged  from  2  to  40  percent,  with  the  average  time  dissatisfied  at  16  percent. 
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Among  Program  Officers,  the  average  percent  time  dissatisfied  with  the  vendor  on  that 
contract  is  3.49  percent.  However,  only  6  percent  of  program  officers  were  dissatisfied  at 
all.  Among  those  dissatisfied,  the  percentage  of  time  dissatisfied  ranged  from  10  tolOO 
percent.  Excluding  the  100  percent  as  an  outlier,  the  overall  percentage  is  reduced  to  1.8 
percent.  Between  Contract  and  Program  Officers  combined,  82.9  percent  were  not 
dissatisfied  at  all  (dissatisfied  0  percent  of  the  time)  with  the  vendor. 


Table  2 

Average  Customer  Satisfaction  Ratings,  Contract  v.  Program  Officers 


Category 

Dissatisfaction 

Promises 

Delivery 

Accomplishment 

Cooperation 

Fair 

price 

Contract 

2.04  % 

9.46 

9.58 

9.62 

9.49 

9.72 

Officer 

Program 

3.49  % 

9.0 

9.32 

9.66 

9.45 

9.58 

Officer 

Correlation 

.82 

.78 

.84 

.82 

.84 

.62 

Difference  in 

No  Sig. 

No  Sig. 

No  Sig. 

No  Sig. 

No  Sig. 

No  Sig. 

Means 

Significance _ 

Difference  in  means  testing  revealed  a  negligible  disparity  between  the  average  program 
officer  response  and  the  average  contract  officer  response  at  all  significance  levels.  The 
correlations  between  contract  and  program  officer  responses  to  the  above  statement  are 
also  generally  very  high,  meaning  that  the  responses  of  the  program  officer  and  the 
contract  officer  are  very  similar  for  each  contract. 
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FINDINGS 

This  analysis  focuses  on  five  criteria  for  addressing  the  objectives  stated  above.  First,  a 
detailed  look  at  the  distribution  of  contracts  by  value,  type,  and  time  will  provide  a 
fundamental  understanding  of  variance  across  contracts.  This  foundation  will  buttress 
findings  in  ensuing  regression  analysis.  Second,  customer  satisfaction  data  analysis  will 
shed  light  on  trends  between  the  six  measures  of  satisfaction  and  contract  data.  In  this 
section,  we  determine  which  variables  influence  customer  satisfaction.  Third,  we  look 
closely  at  total  contract  value  to  find  relationships  with  other  variables  in  the  contract 
data.  This  section  will  offer  the  strongest  comparison  between  contract  groupings 
(explained  below)  and  contract  value.  Fourth,  we  draw  relationships  between  single-bids 
and  other  contract  data  variables.  Analysis  will  reveal  the  factors  that  bring  about  single¬ 
bids  in  our  data  set.  Lastly,  we  analyze  if  discounts  relate  to  any  of  the  contract  data 
variables.  In  this  last  analysis  we  use  overall  vendor  performance  on  past  contracts  as  a 
variable  to  reveal  trends  in  customer-vendor  relationships. 


1.  Distribution  And  Make-up  of  Contracts 

We  separated  60  contracts  into  four  groups — effort  to  get  competition,  hidden 
competition,  single-bid  (from  this  point  forward  in  the  analysis  we  use  this 
interchangeably  with  “one-bid”  due  to  the  single-bid  variable  with  this  name),  and  sole 
source — to  observe  the  effects  on  time  and  value.  Figure  1  below  summarizes  the 
distribution  of  our  sample. 

These  four  group  types  are  dummy  variables,  which  we  further  divide  into  eight  sub¬ 
groupings  to  compare  average  contract  value.  Figure  2  summarizes  average  contract 
value  within  and  across  these  sub-groupings. 

Figure  1 

Distribution  of  Contracts  (n=60) 


Effort  to  get  Hidden  One  Bid  Sole  Source 

Competition  Competion 


Category 
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The  striking  features  at  first  glance  are  the  disparity  in  subgroups.  Awards  with  efforts  to 
get  competition  relative  to  those  without  efforts  to  get  competition  are  higher  in  average 
value  by  $306,000.  One-bid  contracts  are  higher  in  average  value  than  multiple  bid 
awards  by  $269,000,  and  awards  with  hidden  competition  are  $192,000  higher  in  value 
than  awards  without  hidden  competition.  By  contrast,  sole  source  awards  show  no  major 
difference  in  the  sub-grouping. 


Figure  2 

Distribution  of  Contract  Value  by  Sub-group 
(in  thousands) 


Type 


These  observations  can  be  misleading  for  several  reasons.  First,  standard  deviations  of 
these  values  are  very  large,  which  indicate  a  fair  amount  of  outliers  in  the  sample. 
Second,  the  number  within  each  divided  group  is  unequal.  A  quick  look  at  Figure  1  will 
reveal  the  disparity.  70  percent  of  awards  had  some  effort  to  get  competition;  17  percent 
had  some  hidden  competition;  52  percent  had  one  bid,  which  includes  sole  source;  and  35 
percent  were  sole  source.  Lastly,  this  observation  falls  far  short  of  any  determination  that 
says  one-bid  awards,  for  example,  are  higher  in  cost  than  multi-bid  awards.  Factors  such 
as  time,  customer  choice,  and  type  of  work  are  also  relevant. 

Time,  Group,  and  Value 

Time  in  the  award  process  is  weakly  correlated  with  total  value  (.21);  however, 
controlling  for  contract  group  gives  a  fair  judgment  of  what  the  data  groupings  look  like 
when  plotting  time  against  contract  value.  Figure  3  below  is  a  plot  of  all  contracts. 
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Figure  3 

Summary  of  All  Contracts 


Time  (in  weeks) 


Contracts  are  bunched  around  the  $500,000  to  $4  million  value  range.  Several  outliers 
also  show  that  depending  on  groupings  by  type,  average  contract  values  can  be  skewed. 
Figure  4  below  shows  the  separation  of  one-bid  and  multi-bid  contracts.  One-bid  awards 
have  three  outliers  above  the  $14  million  value  range.  Multi-bid  awards  are  consistent 
with  groupings  between  $500,000  and  $8  million. 

Breaking  down  the  one-bid  and  multiple  bid  types  gives  more  detail  as  to  why  values 
may  be  higher  for  the  two  groups  types.  Chart  1  below  breaks  down  the  composition  of 
one-bid  awards. 
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Figure  4 

One  Bid  vs.  Multiple  Bid 
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Incumbent  vendors,  vendors  with  exceptional  past  performance,  or  follow-on  contracts 
make  up  25  percent  of  one-bid  awards;  67  percent  result  from  no  other  bid,  time,  or 
expertise  constraints;  and  8  percent  result  from  customer  choice. 

Chart  1 

Composition  of  One  Bid  Awards 
(excluding  sole  source) 

ED  Incumbent,  Past 
Performance,  or 
Follow  On 

■  No  Other  Bid,  Time  or 
Work  Type  Constraint 

□  Customer  Choice 


Efforts  to  compete  and  hidden  competition  show  another  distribution  of  contract  values 
relative  to  time.  First,  70  percent  of  the  awards  exhibited  some  effort  to  compete,  while 
only  17  percent  had  hidden  competition.  Higher  average  contract  values  of  awards  with 
effort  relative  to  those  without  effort  are  clear  based  on  raw  dollar  value  and  aggregate 
number.  Hidden  competition  shows  similar  trends.  The  scatter  plots  below  in  Figure 
5(a)  and  5(b)  show  these  observations. 
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Figure  5(a) 

Efforts  to  Conpete  Contrac:t 
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Figure  5(b) 
Hidden  Competition 


♦  Hidden 
□  Open 


Finally,  controlling  for  sole  source  contracts  shows  that  these  awards  tend  to  be  in  low 
value,  with  upward  pressure  on  the  average  due  to  outliers.  There  is  one  outlier  upwards 
of  $14  million,  but  the  majority  fluctuates  below  $4  million.  Figure  6  shows  the 
relationship  below.  These  results  depict  a  constant  dollar  amount  for  IT  sole  source 
awards,  as  well  as  a  concentrated  time  of  less  than  9  weeks. 
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Figure  6 
Sole  Source 
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2.  Customer  Satisfaction 

We  arrange  customer  satisfaction  analysis  as  a  series  of  modeled  regressions  for  all 
satisfaction  questions.  Each  question  serves  as  the  dependent  variable  for  separate 
regression  models.  We  seek  evidence  of  some  relationship  between  the  independent 
variables  (primarily  contract  data)  and  customer  satisfaction,  the  strength  of  this 
relationship,  and  their  influence  in  best  value  judgments.  Significance  is  judged  on  .01, 
.05,  and  .1  levels;  however,  coefficients  at  the  .1  levels  should  be  noted  as  weak  in 
statistical  significance. 

Percent  Time  Dissatisfied  With  The  Vendor 

Five  ordinary  least  squares  (OLS)  regressions  were  modeled  for  vendor  dissatisfaction  as 
the  dependent  variable.'*  The  coefficient  on  time  shows  significance  in  Regressions  2  and 

3  at  the  .1  levels,  but  vanishes  in  Regression  5  when  fair  price  is  added.  The  low 
significance  on  time  indicates  no  association  between  time  and  dissatisfaction  level, 
holding  all  other  variables  constant.  Fair  price  alone,  however,  is  significant  (Regression 

4  and  5).  A  one-unit  increase  in  price  satisfaction  rating  is  associated  with  a  decrease  in 
dissatisfaction  level  by  9.43  percent  (.01).  Controlling  for  SOW,  time,  discount,  one-bid 
and  log  of  contract  value,  the  decrease  in  dissatisfaction  rating  associated  with  a  one-unit 
increase  in  price  satisfaction  changes  slightly  to  9.68  percent  (.01).^  Overall  analysis  here 
reveals  very  weak  results,  with  the  exception  being  statistical  significance  only  with  fair 
price.  As  such,  judgment  on  fair  price  is  highly  correlated  to  dissatisfaction  (-0.85);  this 
trend  will  be  consistent  with  other  satisfaction  measures  as  price  judgment  is  a  large  part 
of  satisfaction  in  general. 


See  Appendix  for  complete  data  and  independent  variable  definitions 
^  See  Appendix,  Table  2 
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Table  3 


Summary  of  Significant  Variables  for  Pet  Dissatisfaction  (OLS) 


Dependent  Variable 

Sig.  Regressor 

Sig.  Levels 

Regression 

Pet  Dissatisfaction 

Time 

.1 

2,3 

(0  to  100) 

Vendor  Fair  Price 

.01 

4,5 

Rating  on  Vendor  Keeping  Promises 

Vendor  keeping  promises  as  the  dependent  variable  gave  several  statistically  significant 
results.^  One-bid  as  a  regressor  is  not  significant  until  the  use  of  a  polynomial 
(Regression  3).  This  yields  a  relationship  between  one-bid  and  the  dependent  variable. 
The  predicted  change  in  satisfaction  rating  on  vendors  keeping  promises  associated  with 
a  one-bid  order  relative  to  a  multi-bid  award  on  a  scale  of  1  to  10  is  .79,  holding  discount, 
contract  value,  and  the  square  of  contract  value  constant  (.05).  Leaving  out  contract 
value  in  Regression  4,  and  holding  SOW  and  time  constant,  increases  the  predicted 
change  slightly  to  .84  at  the  .05  level. 

Similarly,  time  and  SOW  also  have  significance  (.05  respectively).  Increases  in  time  (in 
weeks)  of  the  award  process  are  associated  with  a  sli^t  increase  (.11)  in  the  vendor 
keeping  promises  rating;  and  a  one  unit  increases  in  SOW  (on  a  1  to  5  scale)  is  associated 
with  a  .44  increase  in  vendor  keeping  promises. 

Significance  and  magnitude  on  one-bid  remains  unchanged  after  adding  fair  price  and  log 
of  contract  value;  however  time  and  SOW  drop  out  by  holding  these  additional  regressors 
constant.  In  the  last  regression,  a  one-unit  increase  in  fair  price  rating  is  associated  with  a 
.92  increase  in  vendor  keeping  promises  rating,  holding  all  other  regressors  constant 
(.01).’  Both  one-bid  relative  to  multiple  bids  and  increases  fair  price  rating,  are  strongly 
associated  with  satisfaction  ratings  on  vendors  keeping  promises. 


Table  4 

Summary  of  Significant  Variables  for  Vendor  Promises  (OLS) 


Dependent  Variable 

Sig.  Regressor 

Sig.  Levels 

Regression 

Vendor  Promises 

One-bid 

.01,  .05 

3,4,5,6 

(1  to  10) 

Time 

.05,  .1 

4,6 

Statement  of  Work 

.05 

4 

Vendor  Fair  Price 

.01 

5,6 

®  See  Appendix,  Table  3,  OLS  regression  models 

’  See  Appendix,  Table  3,  OLS  regression  models  for  conqjlete  list  of  regressors 
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Rating  on  Vendor  Sticking  to  The  Delivery  Schednle 

We  observe  a  similar  trend  of  statistical  significance  with  one-bid  and  fair  price  while 
using  the  delivery  schedule  rating  (scale  of  1  to  10)  as  the  dependent  variable.  The 
predicted  change  in  delivery  schedule  rating  associated  with  a  one-bid  award  relative  to  a 
multi-bid  award  is  1.03,  holding  discounts  constant  (.05). 

Adding  more  variables  to  the  equation  maintains  the  level  of  significance  on  one-bid 
while  its  magnitude  diminishes.  Adding  time,  SOW,  contract  value,  the  square  of 
contract  value  and  fair  price  shows  the  upward  bias  in  one-bid.  Holding  these  constant, 
the  predicted  change  in  delivery  schedule  rating  associated  with  a  one-bid  award  is  .56 
relative  to  multi-bid  awards  (.05). 

Also  significant  is  fair  price  in  Regression  4  (.01).  A  one-xmit  increase  in  fair  price  rating 
is  associated  with  a  .77  increase  in  delivery  schedule  rating.  Aside  from  one-bid  and  fair 
price,  no  other  variables  were  statistically  significant  at  the  .01  or  .05  level  in  these 
models. 


Table  5 

Summary  of  Significant  Variables  for  Vendor  Delivery  (OLS) 


Dependent  Variable 

Sig.  Regressor 

Sig.  Levels 

Regression 

Vendor  Delivery 

One-bid 

.05 

1,2, 3, 4 

(1  to  10) 

Time 

.1 

3 

Contract  Value 

.1 

3 

Vendor  Fair  Price 

.01 

4 

Rating  on  Vendor  Accomplishing  The  Objective 

Results  are  slightly  different  with  the  vendor  accomplishment  rating  as  the  dependent 
variable.  One-bid  is  only  significant  here  at  the  .1  level,  and  we  fiierefore  ignore  its 
impact  on  vendor  accomplishment.  Vendor  accomplishment  is  important  in  a  best  value 
judgment.  Not  only  does  the  measure  include  price  satisfaction,  but  also  quality  of  work, 
expertise,  timeliness,  product  delivery,  and  cooperation  are  all  very  much  relevant  to  the 
measure.  The  focus  is  therefore  shifted  to  SOW,  contract  value,  time,  and  fair  price.* 

Regression  4  shows  significance  in  both  time  and  SOW  (.05  respectively).  A  one-week 
increase  in  award  process  time  is  associated  with  a  .086  increase  in  vendor 
accomplishment  rating.  Similarly,  an  increase  in  one  unit  of  the  statement  of  work 
quality  rating  is  associated  with  a  .36  increase  in  vendor  accomplishment  rating. 


®  See  Appendix,  Table  5,  OLS  regression  models  for  complete  list  of  regressors 
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These  trends  emphasize  the  importance  of  SOW  quality  and  the  award  process.  A  quality 
SOW  will  leave  little  room  for  incongruence  between  what  the  customer  expects  and 
what  the  vendor  provides.  The  award  process  itself  is  important  if  it  engenders  a 
competition  where  best  value  is  determined.  Regressions  3  and  4  support  this 
relationship. 

Including  fair  price  drops  time  and  SOW  from  the  model.^  A  one-unit  increase  in  fair 
price  rating  is  associated  with  a  .55  increase  in  vendor  accomplishment  rating,  holding 
one-bid,  discount,  time,  SOW  and  contract  value  constant  (.05).  This  might  render  some 
assumptions  ineffective;  however,  we  presume  the  customer  always  negotiates  a  fair 
price,  regardless  of  any  objective.  Here,  we  can  question  whether  it  belongs  in  the  model 
at  all. 


Table  6 

Summary  of  Significant  Variables  for  Vendor  Accomplishment  (OLS 


Dependent  Variable 

Sig.  Regressor 

Sig.  Levels 

Regression 

Vendor  Acconqjlish 

One-bid 

.01 

1,2, 3, 4 

(1  to  10) 

Time 

.05,  .1 

3,4 

Statement  of  Work 

.05 

3,4 

Vendor  Fair  Price 

.01 

5 

Rating  on  Vendor  Cooperation 

The  significance  of  one-bid  is  fairly  consistent  in  this  model  with  vendor  cooperation  as 
the  dependent  variable.  The  predicted  change  in  vendor  cooperation  rating  associated 
with  a  one-bid  award  relative  to  a  multi-bid  award  is  1.06,  holding  discount  and  labor 
hours  contract  constant  (.05).  Holding  a  time  and  materials  contract  constant  brings  the 
significance  level  of  one-bid  down  to  .1,  but  increases  significance  in  time  and  materials. 
The  predicted  change  in  vendor  cooperation  rating  associated  with  a  time  and  materials 
contract  relative  to  labor  hours  and  fixed  price  is  .627,  holding  one-bid  and  discormt 
constant.  Including  the  contract  type  will  help  determine  if  there  is  a  relationship 
between  contract  and  cooperation  under  that  contract.  In  this  model,  statistical 
significance  came  only  from  the  time  and  materials  contract. 

Time  became  significant  after  controlling  for  an  interaction  of  log  of  contract  value  and 
one-bid.  A  one-week  increase  in  award  process  time  is  associated  with  a  .45  decrease  in 
cooperation,  holding  SOW,  discount  and  the  interaction  term  constant  (.05).  A  possible 
explanation  might  come  from  the  relative  importance  of  contracts.  High  profile  contracts 
might  be  scrutinized  more  than  others,  which  could  either  demand  more  from  the  vendor 
or  be  harshly  judged. 


®  See  Appendix,  Table  5,  Regression  5 


24 


U.$.  General  Services  Adminislcallon 


tj  S  A 


Adding  regressors  decreased  the  significance  of  contract  type,  but  maintained 
significance  and  magnitude  for  time.  Fair  price  again  is  significant  with  a  magnitude  of 
.91  (.01). 


Table  7 

Summary  of  Significant  Variables  for  Vendor  Cooperation  (OLS) 


Dependent  Variable 

Sig.  Regressor 

Sig.  Levels 

Regression 

Vendor  Cooperation 

One-bid 

.01,  .05 

1,2, 3,7 

(1  to  10) 

Time 

.05 

4,7 

Statement  of  Work 

.1 

6 

Vendor  Fair  Price 

.01 

7 

Time  and  Materials 

.05,  .1 

2,5,6 

Log  (contract)  x  One- 
bid 

.1 

4 

Rating  on  Vendor  Price  as  Fair  and  Reasonable 

With  fair  price  as  the  dependent  variable,  the  expectation  is  that  discounts  and  multi-bid 
contracts  would  have  significant  results.  On  the  contrary,  the  only  significant  results 
came  in  Regression  4  with  SOW  and  time  (.05  and  .1  respectively).^*^  A  one-unit  increase 
in  SOW  rating  is  associated  with  a  .36  increase  in  fair  price  rating,  holding  one-bid, 
discount,  time,  contract  value,  and  the  square  of  contract  value  constant.  While  SOW 
quality  can  indeed  be  related  to  fair  price,  this  is  a  questionable  result.  In  fact,  it  is  not 
surprising  to  find  almost  no  relationship. 


Table  8 

Summary  of  Significant  Variables  for  Vendor  Fair  Price  (OLS) 


Dependent  Variable 

Sig.  Regressor 

Sig.  Levels 

Regression 

Vendor  Fair  Price 

Time 

.1 

4 

(1  to  10) 

3,4 

Lack  of  convincing  significance  on  time  does  preclude  some  judgment  on  fair  price. 
Figure  3  above  shows  contract  values  for  all  awards  relative  to  time.  The  distribution  is 
fairly  inconclusive  with  higher  average  values  and  standard  deviations  in  the  4  to  9  week 
range  compared  to  a  0  to  3  week  and  plus  9-week  range.  Table  9  notes  the  distribution  of 
averages  and  standard  deviations  for  this  category. 


See  Appendix,  Table  7,  for  complete  data 
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Table  9 


Mean  Contract  Value  Per  Time  Category 
(in  thousands) 


Award  Process  Time  (weeks) 

Average  Contract  Value 

Standard  Deviation 

0to3 

$912 

510 

4  to  9 

$4,087 

5507 

+9 

$3,881 

5083 

Almost  all  contracts  reviewed  were  said  to  have  had  fair  and  reasonable  prices  by 
Contract  and  Program  Officers.  While  it  is  surprising  that  discounts  are  not  significant  at 
any  level,  the  data  reveal  satisfied  customers  regardless  of  discoimts.  One  reason  is  that 
prices  listed  off  the  GSA  schedules  are  considered  fair  and  reasonable,  and  any  additional 
discount  would  only  sweeten  the  deal.  The  strongest  reason  is  that  contract  prices  are 
often  negotiated.  Essentially,  a  contract  is  not  awarded  unless  the  price  is  fair  and 
reasonable.** 


3.  Contract  Value 

Analysis  on  contract  value  is  set  up  as  several  log-linear  OLS  regressions.  Observing  the 
dependent  variable — log  of  contract  value — ^will  show  percent  changes  associated  with 
variations  in  data  we  think  would  most  influence  contract  value. *^  This  analysis  shows 
strong  significance  in  the  influence  of  two  groups  discussed  above:  Effort  to  get 
competition  and  sole  source. 

Contracts  exhibiting  some  effort  to  compete  show  the  strongest  levels  of  significance  at 
.01,  with  one  minor  exception  in  Regression  4  (.05).  This  analysis  shows  very  little 
OVB.  The  predicted  change  in  contract  value  associated  with  efforts  to  compete  relative 
to  no  efforts  is  65  percent,  holding  sole  source  constant  (.01).  Increasing  the  number  of 
variables  held  constant  to  one-bid,  time,  SOW,  and  sole  source,  slightly  decreases  the 
magnitude  of  predicted  change  in  contract  value  to  57  percent. 

An  interesting  relationship  exists  with  sole  source  awards,  which  also  has  strong 
significance.  The  predicted  change  in  contract  value  associated  with  sole  source  awards 
relative  to  non-sole  source  awards  is  53  percent,  holding  efforts  to  compete  constant 
(.05).  There  is  considerable  downward  bias  by  excluding  other  variables,  as  the 
magnitude  increases  with  significance  in  Regressions  4  and  5.  Holding  SOW  and  effort 
to  compete  constant  increases  the  magnitude  of  predicted  change  in  contract  value  to  75 
percent  for  sole  source  awards  relative  to  non-sole  source  awards  (.01).  Including  time 
and  one-bid  in  the  equation  keeps  the  predicted  change  high  at  77  percent  (.05). 

* '  See  Discussion  section  for  more  on  this  area 
See  Appendix,  Table  8,  for  a  complete  list  of  regressors 
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Time  shows  significance  when  controlling  for  one-bid,  SOW,  effort  to  compete,  and  sole 
source  in  Regression  4.  The  coefficient  shows  a  4  percent  predicted  change  in  contract 
value  for  an  additional  week  in  award  process  time  (.05).  The  magnitude  of  change  with 
time  is  not  surprising  given  our  observations  of  average  contract  amount  in  Table  9  and 
Figure  3  above. 


Table  10 

Summary  of  Signifilcant  Variables  for  Log  (Contract  Value)  (Log-Linear  OLS] 


Dependent  Variable 

Sig.  Regressor 

Sig.  Levels 

Regression 

Log  (contract  value) 

Effort  to  Compete 

.01,  .05 

1,2, 3,4,5 

Sole  Source 

.01,  .05,  .1 

1,2,3,4,5 

Time 

.05 

4 

SOW 

.1 

5 

4.  Single-bid  Awards 

Analysis  on  the  dummy  one-bid  variable  is  modeled  as  several  probit  regressions,  and  we 
endeavor  to  identify  factors  that  most  influence  one-bid  awards.  Observing  the  one-bid 
dummy  as  the  dependent  variable  changes  the  interpretation  of  its  relationship  with  the 
regressors  in  each  model.  Here,  we  look  at  differences  in  predicted  probability.*^ 

Awards  with  some  effort  to  compete  show  the  strongest  significance  levels.  All 
coefficients  on  effort  to  compete  are  negatively  correlated  to  one-bid.  That  is,  increased 
efforts  to  compete  are  associated  with  multiple  vendors,  and  decreased  competition  is 
associated  with  evidence  of  one-bid.  This  is  a  likely  observation,  but  interesting  when 
controlling  for  all  other  factors.  The  difference  in  predicted  probability  of  one-bid  is  62 
percent,  given  efforts  to  compete  relative  to  no  efforts,  and  controlling  for  hidden 
competition  (.01).  Adding  more  regressors  decreases  the  difference  in  predicted 
probability  of  one-bid,  but  maintains  a  high  level  of  significance.  Regression  3  shows  the 
difference  in  predicted  probability  of  one-bid  is  49.7  percent,  given  efforts  to  compete 
relative  to  no  efforts,  while  holding  time,  SOW,  hidden  competition,  contract  value,  the 
square  of  contract  value,  and  overall  vendor  satisfaction  rating^'*  constant  (.05). 


See  Appendix,  Table  9,  for  a  complete  list  of  regressors,  and  probabilities 
*'*  Overall  vendor  satisfaction  is  new  to  the  models  and  not  included  as  one  of  the  six  satisfaction  questions. 
It  targets  past  relationships  between  customers  and  vendors.  Small  sample  size  in  this  variable  will  limit 
our  findings.  We  include  it  here  because  of  the  likelihood  that  past  performance  might  have  influence  over 
single-bids.  The  coefficient  is  not  significant,  but  negative.  That  is,  the  more  dissatisfied  the  vendor  is,  the 
greater  the  likelihood  of  competition.  Including  this  variable  is  useful  for  both  log  (contract  value)  and 
discounts  as  dependent  variables. 
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Table  11 

Summary  of  Significant  Variables  for  One-bid  (Probit) 


Dependent  Variable 

Sig.  Regressor 

Sig.  Levels 

Regression 

Log  (contract  value) 

Effort  to  Compete 

.01,  .05 

1,2, 3,4 

(Contract  Value)  ^ 

05,  .1 

3,4 

Hidden  Competition 

.01 

1,2 

5.  Discounts 

The  dependent  dummy  discount  variable  is  used  to  determine  what  factors  might  drive 
prices  down  for  the  customer.  Several  factors  to  consider  are  past  vendor  performance, 
existence  of  competition,  efforts  to  compete,  and  estimated  contract  value.  The 
assumption  is  that  any  of  these  factors  would  increase  the  likelihood  of  a  discount.  The 
results,  however,  are  limiting  due  to  data  constraints.  Some  customer  agencies  thou^t 
this  data  to  be  proprietary  information  or  hindered  access  to  reliable  data. 

Table  12  below  offers  a  glimpse  at  the  limitations  of  the  discoimt  variable  analysis. 
Despite  its  shortcomings,  we  observe  41  awards  with  discounts.  They  are  relatively 
similar  in  distribution  as  the  average  dollar  value  indicates.  Awards  with  no  discount 
have  slightly  lower  value  compared  to  awards  with  discounts.  We  speculate  that  large 
contracts  would  likely  demand  discotmts,  while  smaller  contracts  would  likely  go  without 
a  discount.  A  difference  in  means  test  reveals  no  difference  in  means  between  discount 
and  no  discount  at  all  levels  of  significance.  This  offers  little  conclusive  evidence,  as 
suggested  below,  in  the  ensuing  analysis. 


Table  12 

Distribution  of  Discounts  in  Sample 


Type 

Count 

Frequency 

Average  Contract  Value 

Discount 

19 

.32 

$2,524 

No  Discount 

22 

.37 

$2,290 

Unavailable  Data 

19 

.32 

$3,517 

Total 

60 

1 

The  analysis  shows  time  is  the  only  significant  variable  across  all  four  probit  models.'^ 
With  a  positive  correlation  between  time  and  discount,  controlling  for  no  other  variable, 
the  predicted  probability  of  a  discount,  given  a  2-week  increase  in  award  process  time  is 
7.4  percent  (.05).  Controlling  for  log  of  contract  value  slightly  decreases  the  magnitude 
in  predicted  probability  of  a  discount  to  6.5  percent  (.05).  Both  results  are  statistically 
significant;  however,  when  one-bid  and  overall  vendor  performance  are  included,  time 
drops  out  and  no  variable  is  significant. 


See  Appendix,  Table  10,  for  complete  regression  coefficients  and  significance 
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The  results  here  are  fairly  inconclusive,  and  have  much  to  do  with  the  limitations  of 
obtaining  quality  discount  data.  Significance  on  time,  though,  shows  a  positive  trend 
fi-om  the  sign  on  the  coefficient.  Increased  time  in  the  award  process  is  associated  with 
increase  likelihood  of  discounts.  We  must  be  careful  with  issuing  a  blanket  statement, 
however,  as  customers  do  have  relationships  with  vendors  in  which  they  always  receive 
discounts.  In  such  a  case,  time  is  not  a  factor. 

Table  13 

Summary  of  Significant  Variables  for  Discount  (Probit) 
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DISCUSSION 

Our  findings  reveal  a  number  of  significant  relationships  that  influence  customer 
satisfaction,  price,  single-bids,  and  discounts.  First,  understanding  customer  satisfaction 
is  the  foundation  for  reviewing  best  value  judgments  in  retrospect.  We  know  from 
contract  reviews  that  a  standard  approach  to  best  value  is  judged  on  the  breadth  of 
technical  expertise,  the  extent  of  past  performance  or  corporate  excellence,  personnel 
qualification,  and  price;  and  while  the  weights  assigned  to  each  criterion  vary  by  agency, 
technical  expertise  and  past  performance  usually  rank  above  price.  The  customer 
satisfaction  measures  we  use  point  to  the  effectiveness  of  this  best  value  judgment,  and 
will  facilitate  a  conclusion  on  how  satisfaction  varies  across  single-bid  and  multi-bid 
contracts. 

Second,  price  varies  across  all  contracts.  As  the  data  suggests  above,  there  is  indeed  a 
relationship  between  total  value  and  factors  such  as  time  and  competition.  Our 
observations  therefore  reveal  how  cost  varies  across  single-bid  and  multi-bid  contracts. 

Third,  all  single-bid  awards  (including  sole  source  awards)  are  different  from  multi-bid 
awards.  We  know  the  FAR  mandates  that  RFQs  be  sent  to  at  least  three  vendors  to 
attract  more  than  one  bid  in  the  award  process;  but,  though  not  illegal,  the  process  is 
questioned  when  only  one  bid  is  returned.  What  brings  about  these  single-bid  awards, 
and  how  do  they  compare  to  multi-bid  awards?  Statistical  analysis  reveals  time,  contract 
value,  and  competition  efforts.  Yet,  are  there  additional  biases  not  measured?  We 
discuss  these  factors,  those  not  measured,  and  any  potential  biases  that  misrepresent  the 
data. 

Lastly,  understanding  trends  in  discount  analysis  reveals  the  limitations  of  our  data.  We 
know  that  the  average  discount  is  between  4.2  on  the  low  end  and  5.7  percent  on  the  high 
end;  yet  these  averages  are  largely  inconclusive  due  to  customer  contract  scrubbing  and 
misleading  data. 

We  surmise  our  analysis  by  observing  how  government  satisfaction  changed  over  time. 
Using  Kehnan’s  1990  data  for  comparison,  we  show  marked  improvement  in  satisfaction 
over  time.  The  last  decade  was  as  period  of  procurement  legislation  reform,  and  our 
results  show  improvement. 


CUSTOMER  SATISFACTION 

Customer  satisfaction  is  generally  very  high.  Despite  a  few  outliers  in  the  sample,  we 
observe  low  dissatisfaction  levels  and  very  high  ratings  on  accomplishment,  delivery, 
cooperation  and  prices.  One  factor  ubiquitous  in  significant  relationship  with  customer 
satisfaction  measures  is  the  fair  and  reasonable  price  rating.  This  is  not  surprising  given 
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strong  correlations  between  price  fairness  and  other  measures;  however,  it  is  evidence  of 
the  relative  importance  of  price  in  satisfaction  judgments  (Table  14  below). 

Table  14 


Correlations  Between  Fair  Price  Ratings  and  Other  Satisfaction  Variables 


Variables 

Dissatisfaction 

Promises 

Delivery 

Accomplishment 

Cooperation 

Fair  Price 

-0.85 

0.846 

0.76 

0.705 

0.826 

Award  process  time  bears  some,  albeit  weak,  significance  to  satisfaction,  and  appears  as  a 
small  factor  in  each  measure.  Aside  from  the  rating  on  fair  price,  award  time  is  the  only 
other  significant  factor  in  dissatisfaction  levels.  In  this  case,  increased  award  process 
time  decreased  the  dissatisfaction  level.  Among  other  factors,  time  increased  in 
significance  on  vendor  promise  ratings,  accomplishment,  and  cooperation.  Increased 
time  is  usually  associated  with  increased  competition,  in-depth  best  value  judgments,  and 
price  negotiation;  yet,  it  is  interesting  that  increased  time  is  associated  with  small 
decreases  in  vendor  cooperation  ratings  while  all  other  ratings  have  the  opposite  effect. 
The  likely  explanation  is  that  large  contracts  decrease  the  cooperation  level  of  vendors, 
perhaps  due  to  greater  room  for  error  and  resistance  to  customer  demands.  Indeed,  while 
contract  value  is  not  a  significant  variable  in  the  cooperation  model,  it  has  a  weak 
negative  correlation  with  cooperation. 

While  contract  cost  had  no  significance,  one-bids  show  relatively  high  significance  levels 
across  measures  on  vendor  promises,  delivery,  accomplishment,  and  cooperation  ratings. 
That  is,  on  each  measure  the  predicted  change  in  rating  is  both  positive  and  higher  for 
single-bid  awards  than  multi-bid  awards.  This  is  a  strange  reversal  of  expected  trend,  as 
we  predicted  the  opposite  to  be  the  case.  There  are  three  reasons  that  offer  explanations 
to  this  relationship.  First,  one-bids  includes  the  sole  source  measure.  Sole  source  work  is 
largely  the  result  of  follow-on  contracts,  incumbency,  or  specialty  of  services.  Chart  2 
below  shows  the  division  of  one-bid  awards. 

Chart  2 

Composition  of  One  Bid  Awards 
(includes  sole  source  awards) 

E  Follow  On/Incumbent 
■  Timing/Type 

□  History 

□  Expertise 


3%  6% 


Follow-on  and  specialty  of  services  makes  up  50  percent  of  one-bid  contracts.  These 
vendors  provide  services  that  no  other  can,  or  provide  a  service  as  logical  follow-on  to 
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work  already  in  progress.  The  likely  effect  is  high  satisfaction  because  the  service  is 
being  provided  regardless  of  few  competing  alternatives. 

Second,  pricing  may  be  more  accurate  for  single-bid  awards  than  for  multi-bid  awards. 
Since  price  competition  may  not  exist,  the  submission  of  additional  costing  data  could 
allow  for  more  complete  cost  and  price  evaluations  for  the  government.  Satisfaction  with 
this  price  could  drive  this  hi^er  trend  in  satisfaction  ratings.  This  explanation  is  offset 
slightly  when  looking  at  the  relationship  between  one-bid  and  fair  vendor  prices.  One- 
bid  shows  no  significance;  yet,  this  is  offset  with  little  or  no  significance  of  any  variable 
on  this  rating.  Rather,  judgments  on  fair  vendor  prices  appear  to  be  high  regardless  of 
any  factor. 

Third,  the  data  is  slanted  in  favor  of  single  bids.  Table  15  below  summarizes  this  trend. 
On  average,  single-bid  and  sole  source  awards  have  lower  dissatisfaction  ratings  than 
multi-bid  awards.  Similarly,  they  have  higher  satisfaction  ratings  in  all  other  categories. 


Table  15 

Average  Ratings  by  Category 


Category 

Dissatisfaction 

Promises 

Delivery 

Accomplishment 

Cooperation 

Fair  price 

Total 

Average 

1.47 

9.6 

9.6 

9.79 

9.7 

9.7 

One-bid 

Std  Dev 

5.14 

.67 

.82 

.57 

.67 

.78 

One-bid, 

Average 

.30 

9.9 

10 

9.9 

9.9 

9.8 

No  Sole 

Source 

Std  Dev 

.75 

.28 

0 

.27 

.27 

.55 

Sole 

Average 

2.26 

9.52 

9J9 

9.7 

9.66 

9.64 

Source 

Std  Dev 

6.52 

.79 

.988 

.69 

.81 

.91 

Multi- 

Average 

5.78 

8.97 

9 

9.34 

9.13 

9.49 

bid 

Std  Dev 

18.1 

1.87 

1.67 

1.61 

1.8 

1.58 

The  explanation  lies  in  the  standard  deviations  of  each  category.  Dissatisfaction  has  a 
markedly  higher  standard  deviation  for  multi-bid  awards  than  single  or  sole  source  bids. 
This  trend  is  consistent  across  each  other  category  as  well.  Of  the  contracts  reviewed, 
three  out  of  the  four  total  outliers  are  multi-bid  and  high  in  dissatisfaction;  and  when 
Contract  and  Program  Officers  were  both  included  in  the  sample,  it  doubled  the  number 
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of  outliers.  These  vendors  garnered  dissatisfaction  ratings  of  between  50  and  100 
percent,  skewing  the  multi-bid  satisfaction  rating  in  the  process. 

We  have  labeled  the  poor  performing  contracts  as  outliers;  but  it  can  certainly  be  the  case 
that  a  larger  majority  of  multi-bid  awards  provide  poor  service  compared  to  single-bid 
awards.  If  this  were  the  case,  other  factors  such  as  close  scrutiny  of  single-bid  vendors 
(motivating  these  vendors  to  provide  the  best  service)  would  come  into  play.  These 
observations  do  not  mean  we  commit  to  the  extreme  and  say  all  single-bids  are  great  for 
government  business.  Rather,  we  are  reluctant  to  condemn  single-bid  awards  based  on 
our  observations. 

The  SOW^^  is  another  point  of  significance  meriting  discussion  on  best  value.  It  shows 
relevancy  in  vendors  keeping  promises,  accomplishment,  and  prices.  Specifically,  an 
increase  in  the  SOW  quality  rating  is  associated  with  an  increase  in  customer  satisfaction. 
This  trend  is  expected,  as  several  Contract  Officers  echoed  problems  with  SOW  quality 
during  the  award  process.  Sloppy  or  vague  SOWs  leave  the  vendor  with  an  incomplete 
or  incoherent  set  of  guidelines  for  the  contracted  work.  As  such,  SOWs  of  lesser  quality 
increase  prices  and  decrease  work  quality.  Our  data  supports  this  trend.  In  faet, 
increasing  the  quality  rating  of  SOWs  by  one  point  on  a  scale  of  one  to  five  increases  the 
three  ratings  by  .4  points.  With  already  high  average  ratings  above  9.4  on  satisfaction, 
this  small  change  would  result  in  a  relatively  large  increase.  Given  the  nature  of  best 
value  judgments,  it  is  in  the  best  interest  of  each  customer  to  improve  or  keep  high 
standards  for  the  SOW.  If  work  specifications  are  poorly  written,  best  value  becomes 
less  relevant. 


CONTRACT  VALUE  (PRICE) 

We  see  in  the  findings  above  that  contract  value  varies  across  different  dimensions.  Our 
findings  show  the  strongest  variables  associated  with  changes  in  price  to  be  time,  efforts 
to  compete,  and  sole  source  awards.  Not  surprisingly,  sole  source  awards  are  very 
significant  in  each  analysis  of  contract  value.  That  is,  the  predicted  change  in  contract 
value  associated  with  sole  source  awards  is  higher  (anywhere  between  53  to  75  percent 
higher)  than  non-sole  source  awards.  Tempting  as  it  may  be  for  customers  to  pass  this 
off  as  vendor  expertise  or  a  follow-on  contract,  those  reasons  account  for  only  one 
portion  of  sole  source  awards.  For  example  we  know  that  33  percent  of  sole  source 
awards  have  some  relation  to  timing  constraints;  and  in  five  cases  customers  failed  to 
allocate  more  than  four  days  for  the  awards  process.  Compared  to  multi-bid  awards  that 
averaged  6.6  weeks,  sole  source  awards  averaged  2.8  weeks  in  award  process  time.’^ 
This  is  a  reasonable  area  for  improvement.  The  reluetance  to  compete  contracts  based  on 
a  need  for  quiek  turnaround  time  on  contract  renewals  is  a  poor  excuse  for  urgency. 


The  average  SOW  rates  4.34  with  a  range  of  2  to  5 
’’  The  average  time  to  get  under  contract  is  5.58  weeks  with  a  range  of  1  day  to  37  weeks 
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Customers  engaging  in  advanced  planning  could  eliminate  the  higher  costs  associated 
with  these  single-bids. 

The  alternative,  and  possible  explanation,  to  this  argument  is  that  sole  source  awards  are 
predicted  to  be  more  expensive  because  they  require  specific  tasks  and  expertise  that  no 
other  vendor  provides.  In  60  percent  of  sole  source  cases,  for  example,  vendor  specific 
IT  systems  guided  sole  source  need  for  contract  renewal.  This  would  call  for  urgency  in 
the  renewal  process. 
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Figure  7 

Average  Contract  Values  Across  Contracts 


(excludes  1  outlier  on  multi-bid) 

$2,851 


Type 

Figure  7  shows  the  average  dollar  value  of  contracts  by  type.  These  averages  support  the 
findings  above  that  the  predicted  change  in  contract  value  is  higher  for  sole  source 
awards  than  non-sole  source  awards.  We  chose  to  exclude  one  outlier  on  contract  value 
above  $20  million  for  the  multi-bid  group,  as  it  inflated  the  true  average.  We  also 
observe  single-bid  contracts  to  be  slightly  higher  in  average  value  than  multi-bid;  this  is 
largely  due  to  the  low  number  of  single-bids  in  the  sample,  in  which  any  outliers  inflate 
its  average  relative  to  a  larger  group  with  a  more  even  distribution.  Again,  this  is  not 
surprising  as  some  contracts  are  much  larger  due  to  scope,  type  of  work,  and  are  sparing 
in  number.  One  concern  is  that  customers  create  fiiture  sole  source  situations  for 
themselves  by  entering  into  original  sole  source  contracts.  Such  vendors  subsequently 
use  unique  software  and  hardware  that  would  manifest  into  high  transition  costs  if 
services  were  recompeted. 

Time  is  related  to  contract  value.  A  one-week  increase  in  the  award  process  is  associated 
with  a  4.6  percent  increase  in  contract  value.  In  this  case,  it  is  important  to  note  that 
correlation  is  not  the  same  as  causation.  Time  does  not  increase  the  contract  value; 
rather,  larger  contracts  may  increase  the  award  process  to  allow  for  competition.  As 
such,  efforts  to  compete  were  associated  with  a  predicted  change  in  contract  value  of 
between  57  and  69  percent  relative  to  no  efforts  to  compete  -  the  higher  the  value,  the 
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more  likely  the  search  for  competition.  Among  the  other  variables  in  best  value,  this  is 
evidence  that  customers  indeed  try  to  obtain  a  best  value  judgment  on  the  price  of  larger 
contracts. 


SINGLE-BIDS 

Some  discussion  above  argues  that  decreases  in  the  award  process  time  increases  the 
likelihood  of  sole  source  and  single-bid  awards;  however,  while  increased  time  is 
associated  with  increased  competition  (time  is  negatively  correlated  to  the  one-bid 
variable),  it  bears  no  significance  in  our  findings.  This  could  be  due  to  the  low  number  of 
single-bid,  non-sole  source,  contracts  in  our  sample.  We  are  therefore  cautious  to  make 
any  blanket  statement  regarding  time  and  single-bid,  but  recommend  a  larger  sample  of 
single-bid,  non-sole  source,  contracts  in  future  studies. 

The  analysis  reveals  efforts  to  compete,  hidden  competition,  and  contract  value  as 
significant  factors  associated  with  single-bid  awards.  The  difference  in  predicted 
probability  of  single-bid  awards  is  between  49  and  62  percent  for  awards  with  evidence 
of  efforts  to  compete  relative  to  no  effort,  with  effort  being  negatively  correlated  with 
single-bids.  That  is,  more  efforts  to  compete  are  associated  with  increased  competition. 
This  is  an  expected,  but  important,  outcome;  by  competing  services,  customers  will 
decrease  the  probability  of  single-bid  outcomes  by  50  percent  or  more. 

Another  expected  trend  is  the  significance  of  hidden  competition.  Evidence  of  hidden 
competition  is  associated  with  increased  probability  of  single-bid  contracts.  This  is  on 
the  order  of  the  same  magnitude  as  efforts  to  compete.  While  this  provokes  the  obvious 
recommendation  calling  for  increasing  efforts  to  ensure  vendors  are  not  dissuaded  fi-om 
competing,  the  small  number  of  hidden  competition  awards  (10  of  60)  precludes  a  clear 
judgment  to  this  effect.'*  The  main  difficulty  here  is  vendors  taking  themselves  out  of 
the  bid  due  to  competing  large  or  well-known  firms.  To  this  effect,  customers  can  reduce 
the  impact  of  hidden  competition  through  policies  that  should  already  be  in  practice.  For 
example,  responsible  agencies  will  try  to  compete  large  businesses  with  other  large 
businesses,  small  businesses  with  other  small  businesses,  or  if  mixed,  give  small 
businesses  some  advantage  in  the  bidding  process  to  ensure  fair  competition. 

There  is  no  relationship  between  single-bids  and  contract  value.  While  the  ubiquitous 
concern  in  government  contracting  is  that  single-bids  are  higher  in  value,  there  is  no 
significance  on  this  variable.  Much  of  this  is  due  to  very  large  contracts  that  sway  the 
data  for  both  single-bid  and  multi-bid  contracts.  Essentially,  some  contracts  are 
inherently  larger  than  others  due  to  the  nature  and  scope  of  the  work.  Five  contracts  had 
this  characteristic,  which  inflated  averages  on  both  sides. 


**  When  we  controlled  for  contract  value  significance  in  this  variable  drops  out. 
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DISCOUNTS 

The  only  measured  factor  with  significance  determining  discounts  is  time;  however,  we 
question  this  result  for  two  reasons.  First,  the  difference  in  predicted  probability  of 
discounts,  given  a  2-week  change  in  award  process  time  is  7.4  percent  -  a  low 
probability.  While  the  relationship  is  positive — an  increase  in  award  time  increases  the 
likelihood  of  discoxmts — adding  additional  variables  drops  time  from  significance. 
Second,  the  data  on  discoxmts  is  highly  questionable;  notwithstanding  limited  access  to 
discoimt  information,  several  immeasurable  factors  influence  discounts.  We  know  fi'om 
interviews  that  if  discounts  are  offered,  it  is  usually  for  three  reasons:  Vendors  want  to 
continue  work  on  government  contracts  in  the  future;  vendors  have  a  history  with 
government  contracts  and  customers  in  the  past;  and  vendors  think  that  others  are 
competing  for  the  award  and  want  the  most  attractive  bid. 

Because  time  is  positively  associated  with  larger  contract  values,  the  higher  predicted 
change  in  contract  value  for  time  increases  does  offer  some  imderstanding  of  an 
underlying  trend  in  discounts.  It  implies  that  highly  scrutinized  contracts  may  take  more 
time  and  attract  discounts  because  these  awards  are  so  critical  for  future  business 
opportunities  with  the  government.  If  this  is  the  case  indeed,  then  our  observations  are 
highly  relevant. 


SATISFACTION  CHANGE  OVER  TIME 

In  1990,  Dr.  Steve  Kelman  of  the  John  F.  Kennedy  School  of  Government  conducted  a 
study  of  the  federal  procurement  system.  Part  of  this  effort  asked  government  workers  to 
state  in  percentage  terms  their  dissatisfaction  level,  and  their  degree  of  satisfaction  with 
vendors  on  a  scale  of  1  to  10  (1  being  worst,  10  being  best)  in  terms  of  vendors  keeping 
promises  and  sticking  to  the  contracted  delivery  schedule.  Contract  and  Program 
Officers  were  asked  these  same  questions  fourteen  years  later  for  this  study  on  single-bid 
IT  service  contracts. 

In  1990  government  respondents  indicated  a  24.5  percent  dissatisfaction  level  with  the 
vendor  in  general.  Responding  to  the  same  question  today,  interviewees  indicate  a  4.77 
percent  dissatisfaction  level  with  the  vendor.  On  a  scale  of  1  to  10,  Kelman  reported  an 
average  7.0  rating  for  vendors  keeping  promises  according  to  government  respondents  in 
1990.  Today,  Contract  and  Program  Officers  rate  the  vendor  on  keeping  promises  at  an 
average  of  9.3  on  a  10-point  scale.  Similarly,  Kelman  asked  government  respondents  to 
rate  vendors  in  terms  of  sticking  to  the  contracted  delivery  schedule.  The  average  rating 
on  a  10-point  scale  in  1990  was  6.2.  With  the  same  question  today,  contracting  officers 
responded  with  an  average  rating  of  9.3. 

Vendor  performance  improved  substantially  over  time,  undoubtedly  due  to  a  combination 
of  new  legislation  in  the  1990s,  greater  customer  responsibility,  and  a  greater  demand  for 
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quality  services.  The  private  sector  role  in  government  contracting  has  become  more 
important  over  time,  sending  a  message  to  vendors  that  if  quality  and  prices  are  not  up  to 
par,  customers  will  not  be  afraid  to  recompete  services.  The  role  of  Contract  Officers  has 
also  changed  over  time,  as  the  FAR  has  given  them  leeway  to  operate  on  their  best 
business  judgment.  As  such,  members  of  the  procurement  team  may  use  their  best 
judgment  if  a  specific  strategy,  practice,  policy  or  procedure  is  in  the  best  interests  of  the 
government.  These  reforms  have  improved  Contract  Officer  discretion  in  the  process, 
and  facilitated  improved  best  value  judgments. 
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CONCLUSION 


Our  study  examined  the  verity  of  the  common  perception  that  single-bid  orders  are  more 
costly,  lower  in  performance  quality,  and  shorter  in  pre-award  process  time  than 
competitively-bid  orders.  In  short,  we  found  that  single-bid  orders  are  not  more  costly, 
are  not  lower  in  performance  quality,  and  that  award  process  time  is  not  related  to  the 
number  of  bids.  The  main  conclusions  are  outlined  below. 

First,  our  findings  reveal  that  customer  satisfaction  is  very  high  for  all  types  of  orders. 
We  found  low  levels  of  dissatisfaction  with  vendors,  and  high  levels  of  satisfaction  in 
accomplishment  of  the  desired  objective,  keeping  promises,  sticking  to  the  delivery 
schedule,  and  fairness  of  prices.  Since  these  satisfaction  measures  are  a  good  proxy  for 
whether  or  not  best  value  is  actually  an  effective  way  of  selecting  vendors,  we  can  say 
with  certainty  that  the  current  best  value  approach  is  getting  customers  what  they  need  in 
a  way  advantageous  for  all  involved. 

It  is  also  the  case  that  customer  satisfaction  has  improved  over  time.  Compared  with  the 
findings  from  Kelman’s  study,  our  results  show  that  the  procurement  process  has 
improved  tremendously  over  the  past  fourteen  years. 

SOWs  have  a  marked  impact  on  customer  satisfaction.  It  is  their  role  to  describe  the 
work  to  be  done  in  detail;  to  this  effect,  the  more  instruction  received,  the  better  the 
vendor  performance  in  filling  the  needs  of  the  customer. 

In  contrast  to  conventional  wisdom,  the  number  of  bids  received  on  a  given  order  does 
not  affect  satisfaction.  Analysis  shows  that  a  larger  proportion  of  competitively-bid 
contracts  were  poor-performing  relative  to  single-bid  contracts.  However,  it  is  important 
to  note  that  this  could  be  due  to  some  abnormality  in  data  selection;  it  is  hard  to 
extrapolate  fi-om  our  study  due  to  its  size  and  location  constraints.  For  more  complete 
answers,  we  recommend  a  study  with  larger  sample  sizes,  a  broad  selection  of  agencies 
and  contract  officers  in  different  parts  of  the  country,  and  mandatory  agency 
participation. 

Second,  price  varies  across  all  contracts,  regardless  of  the  number  of  bids.  Multi-bid  and 
single-bid  awards  have  similar  average  values,  depending  on  the  inclusion  of  outliers. 
Sole  source  awards,  however,  tend  to  be  higher  in  value.  Tempting  as  it  may  be  to  pass 
this  off  as  vendor  expertise  or  a  follow-on  contract,  these  reasons  account  for  only  one 
portion  of  sole  source  awards.  Time  constraints  also  played  a  role;  compared  to  multi-bid 
awards  that  averaged  6.6  weeks,  sole  source  awards  averaged  2.8  weeks  in  award  process 
time. 

In  some  situations,  the  decisions  made  by  customers  themselves  put  them  in  the  position 
of  accepting  sole  source  awards  into  the  foreseeable  future.  In  the  IT  field,  so  many 
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awards  serve  unique  software  and  systems.  Not  competing  an  original  award  can  have 
ramifications  into  the  future,  as  all  updates  to  that  software  will  likely  have  to  be  made  by 
the  original  developer  of  the  product.  Since  we  also  discovered  that  there  is  a 
relationship  between  efforts  to  compete  and  increased  competition,  it  is  all  the  more 
important  to  allow  for  adequate  competition  time  in  the  original  contract.  If  the  effort  is 
made,  it  is  likely  that  competition  will  result. 

There  are  some  trends  in  which  contracts  receive  discounts  and  which  do  not.  We  know 
that  the  average  discount  was  between  4.2  and  5.7  percent;  yet  we  hesitate  to  draw 
conclusions  here  because  of  incomplete  data.  Longer  award  process  time  increases  the 
likelihood  of  receiving  a  discoimt,  and  increased  time  allows  for  increased  negotiation. 
This  trend  is  consistent  with  what  we  had  assumed.  We  discovered  that  discounts  were 
offered  by  three  types  of  vendors:  Vendors  who  hoped  to  continue  working  with  the 
government  in  the  future,  vendors  who  had  worked  with  the  government  in  the  past,  and 
vendors  who  thought  other  vendors  might  be  competing  for  the  award. 

To  conclude,  we  reiterate  the  findings  of  our  study.  Single-bid  orders  are  not  more 
costly,  are  not  lower  in  performance  quality,  and  award  process  time  is  not  related  to  the 
number  of  bids.  Customer  satisfaction  has  improved  over  time,  as  the  best  value  system 
under  the  new  regulations  appears  to  be  working.  Although  we  advocate  consideration  of 
the  recommendations  we  have  proposed,  overall  we  are  optimistic  about  the  effect  of  the 
recent  reforms  made  to  the  procurement  system. 
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APPENDICES 


Table  1 


Variables  in  Regression  Models  of  Customer  Satisfaction  and  Contract  Data 


Variable 

Definition 

Sample  Average 

Customer  Satisfaction  Data  Variables 

Dissatisfaction 

Customer  rating  on  percent  level  of 
dissatisfaction  with  the  vendor  on 
the  contract  in  question  (0  to  100% 
scale,  0  =  completely  satisfied;  100 
=  completely  dissatisfied) 

3.63 

Promises 

Customer  rating  for  vendor 
performance  on  keeping  promises  (1 
to  10  scale,  1  =  worst;  10  =  best) 

9.38 

Delivery 

Customer  rating  on  the  vendor 
sticking  to  the  contracted  delivery 
schedule  (1  to  10  scale,  1  =  worst; 
10  =  best) 

9.32 

A  ccomplishment 

Customer  rating  on  the  vendor 
accomplishing  what  was  expected  (1 
to  10  scale,  1  =  worst;  10  =  best) 

9.58 

Cooperation 

Customer  rating  for  vendor 
performance  on  keeping  promises  (1 
to  10  scale,  1  =  worst;  10  =  best) 

9.46 

Vendor  Fair  Price 

Customer  rating  for  vendor 
performance  on  keeping  promises  (1 
to  10  scale,  1  =  worst;  10  =  best) 

9.59 

Vendor  Overall 

Customer  rating  on  percent  level  of 
dissatisfaction  with  the  vendor  in 
general  (0  to  100%  scale,  0  = 
completely  satisfied;  100 
completely  dissatisfied) 

4.77 

Contract  Data  Variables 

Contract  value 

Total  value  of  contract 

$2,797,488 

One-bid 

1  if  award  was  single-bid  (includes 
sole  source) 

0  otherwise 

.51 

Sole  Source 

1  if  sole  source  award 

0  otherwise 

.358 

Discount 

1  if  award  had  discount 

0  otherwise 

.46 

Time  in  Weeks 

Length  of  time  to  get  under  contract 
in  weeks 

5.58 

SOW 

Customer  rating  on  statement  of 
work  (1  to  5  scale,  1  =  worst;  5  = 
best) 

4.34 

Labor  Hours 

1  if  contract  is  labor  hours 

0  if  otherwise 

.164 

Time  and  Materials 

1  if  contract  is  time  and  materials 

0  otherwise 

.443 

Fixed  Price 

1  if  contract  is  fixed  price 

0  otherwise 

.417 

Effort  to  compete 

1  if  some  effort  to  compete 

0  otherwise 

.696 

Hidden  Competition 

1  if  hidden  competition  present 

0  otherwise 

.157 
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Table  2 


Dissatisfaction  Regressions 


Dependent  Variable:  Percent  time  customer  is  dissatisl 

ied  with  the  vendor 

Regression 

(1) 

(2) 

(3) 

(4) 

(5) 

Regression  Model: 

OLS 

OLS 

OLS 

OLS 

OLS 

Log  (contract  value) 

nm 

MliMM 

■cMh 

One-bid 

Kigiii 

mmoM 

mBM 

Discount 

mIMM 

Time  in  Weeks 

mEBM 

SOW 

HUH 

■nnH 

Contract  Value 

6.65e'> 
(4.6  le®) 

(Contract  Value)  ^ 

-3.50e'^ 

(2.46e’'^) 

Vendor  Price  Fair 

mBmm 

Constant 

mfmM 

■iE5h 

N 

60 

52 

52 

60 

52 

R^ 

.062 

.133 

.243 

.729 

.816 

These  regressions  were  estimated  using  the  PAE  data  set  described  in  Research  Methods. 


Standard  errors  are  given  in  parentheses  under  the  coefficients.  Individual  coefficients  are 
statistically  significant  at  the  *10%,  **5%  or  ***1%  level. 
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Table  3 

Promise  Satisfaction  Regressions 


Dependent  Variable:  Rating  on  vendor  keeping  promises 


Regression 

(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

Regression  Model: 

OLS 

OLS 

OLS 

OLS 

OLS 

OLS 

Log  (contract  value) 

-.150 

(.21) 

■m 

One-bid 

■lyn 

mmm 

WBBm 

mmm 

Discount 

R^i 

mkiim 

mmSM 

mmsm 

Time  in  Weeks 

jWliUM 

■m 

SOW 

RRI 

Contract  Value 

-4.43e'' 

(3.74eA 

(Contract  Value)  ^ 

IKI 

Vendor  Price  Fair 

' 

■Eoni 

Constant 

mmm 

mBSm 

■HgUl 

■in 

Km 

N 

60 

52 

52 

52 

53 

52 

R^ 

.06 

.012 

.200 

.322 

.785 

.785 

These  regressions  were  estimated  using  the  PAE  data  set  described  in  Research  Methods. 


Standard  errors  are  given  in  parentheses  under  the  coefficients.  Individual  coefficients  are 
statistically  significant  at  the  *10%,  **5%  or  ***1%  level. 
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Tabled 

Delivery  Satisfaction  Regressions 

Dependent  Variable:  Rating  on  vendor  sticking  to  the  delivery  schedule 


Regression 

(1) 

(2) 

(3) 

(4) 

Regression  Model: 

OLS 

OLS 

OLS 

OLS 

Log  (contract  value) 

One-bid 

Discount 

Time  in  Weeks 

SOW 

Contract  Value 

(Contract  Value)  ^ 

MlUlH 

8.76e’‘" 

(1.29e-'^) 

Vendor  Price  Fair 

Constant 

mBm 

N 

54 

54 

52 

52 

R^ 

.114 

.12 

.25 

.645 

These  regressions  were  estimated  using  the  PAE  data  set  described  in  Research  Methods. 


Standard  errors  are  given  in  parentheses  under  the  coefficients.  Individual  coefficients  are 
statistically  significant  at  the  *10%,  **5%  or  ***l%  level. 
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Table  5 

Accomplishment  Satisfaction  Regressions 


Dependent  Variable;  Rating  on  vendor  accomplishing  the  objective 


Regression 

(1) 

(2) 

(3) 

(4) 

(5) 

Regression  Model: 

OLS 

OLS 

OLS 

OLS 

OLS 

Log  (contract  value) 

mBM 

-.496 

(.313) 

One-bid 

■sSh 

mBEM 

n 

Discount 

mm 

Time  in  Weeks 

^91 

Hill 

SOW 

mm 

Contract  Value 

-6.59e'' 

(3.65e-) 

-3.41e' 

(3.61eG 

(Contract  Value)  ^ 

3.406*''^ 

(1.96e'''') 

fm 

Vendor  Price  Fair 

KH||« 

mBSM 

Constant 

g  39*** 

(•538) 

13.4*** 

(3.08) 

13.98*** 

(3.47) 

8.11*** 

(.689) 

3.52*** 

(1.32) 

N 

54 

53 

52 

52 

52 

R^ 

.09 

.143 

.21 

.329 

.58 

These  regressions  were  estimated  using  the  PAE  data  set  described  in  Research  Methods. 


Standard  errors  are  given  in  parentheses  imder  the  coefficients.  Individual  coefficients  are 
statistically  significant  at  the  *10%,  **5%  or  ***1%  level. 
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Table  6 

Cooperation  Satisfaction  Regressions 
Dependent  Variable:  Rating  on  vendor  cooperation  with  the  government 


Regression  (1)  (2)  (3)  (4)  (5)  (6) 


Regression  Model:  OLS  OLS  OLS  OLS  OLS  OLS 


Log  (contract  value) 


One-bid 


Discount 


Time  in  Weeks 


SOW 


Contract  Value 


(Contract  Value) 


Vendor  Price  Fair 


Labor  Hours 


Time  and  Materials 


Fixed  Price 


Log  (contract  value)  x  One-bid 


Constant 


N 


55 

55 

55 

53 

53 

53 

53 

.114 

.146 

.148 

.146 

.179 

.25 

.762 

These  regressions  were  estimated  using  the  PAE  data  set  described  in  Research  Methods. 
Standard  errors  are  given  in  parentheses  under  the  coefficients.  Individual  coefficients  are 
statistically  significant  at  the  *10%,  **5%  or  ***1%  level. 
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Table  7 

Price  Satisfaction  Regressions 


Dependent  Variable;  Rating  on  vendor  price  as  fair  and  reasonable 


Regression 

(1) 

(2) 

(3) 

(4) 

Regression  Model: 

OLS 

OLS 

OLS 

OLS 

Log  (contract  value) 

One-bid 

HBH 

Discount 

Time  in  Weeks 

■BH 

SOW 

Contract  Value 

hSsH 

(Contract  Value)  ^ 

2.76e-“' 

t2.20e'‘'') 

Vendor  Price  Fair 

Sole  Source 

mEEsSMm 

Constant 

mwmssm 

HRMIIIH 

N 

60 

53 

52 

53 

R" 

.047 

.08 

.139 

.208 

These  regressions  were  estimated  using  the  PAE  data  set  described  in  Research  Methods. 


Standard  errors  are  given  in  parentheses  imder  the  coefficients.  Individual  coefficients  are 
statistically  significant  at  the  *10%,  **5%  or  ***1%  level. 
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Table  8 


Contract  Value  Regressions 
Dependent  Variable:  Log  (Total  Contract  Value) 


Regression 

(1) 

(2) 

(3) 

(4) 

(5) 

Regression  Model: 

OLS 

OLS 

OLS 

OLS 

OLS 

One-bid 

— 

Discount 

Time  in  Weeks 

SOW 

UBSmm 

Effort  to  Compete 

m^SSSm 

Sole  Source 

■Bn 

Constant 

mBSM 

N 

60 

60 

60 

60 

60 

R^ 

.0532 

.0537 

.089 

.136 

.14 

These  regressions  were  estimated  using  the  PAE  data  set  described  in  Research  Methods. 
Standard  errors  are  given  in  parentheses  under  the  coefficients.  Individual  coefficients  are 
statistically  significant  at  the  *10%,  **5%  or  ***1%  level. 
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Table  9 

Single-bid  Regressions 


Dependent  Variable:  One-bid 

=  1  if  single- 

bid,  =  0  if  competition 

Regression 

(1) 

(2) 

(3) 

(4) 

Regression  Model: 

Probit 

Probit 

Probit 

Probit 

Discount 

Time  in  Weeks 

BESIB 

SOW 

Effort  to  Compete 

Sole  Source 

Hidden  Competition 

iRsm 

Contract  Value 

-8.07e-“ 

(7.27e-’) 

-1.66e'' 

(9.30e'®) 

-1.1  le' 
(1.22e'’) 

Vendor  Overall 

■BShI 

Constant 

iungmiii 

Difl'erence  in  Predicted  probability  of  one- 
bid,  effort  to  compete  v.  no  efl'ort  (percentage 
points)*’ 

62% 

61.8% 

48.8% 

49.7% 

These  regressions  were  estimated  using  the  PAE  data  set  described  in  Research  Met! 

ilods. 

Standard  errors  are  given  in  parentheses  under  the  coefficients.  Individual  coefficients  are 
statistically  significant  at  the  *10%,  **5%  or  ***1%  level. 


Effort  to  get  competition  is  negatively  correlated  with  one-bid.  Increased  efforts  to  get  competition  are 
associated  with  multiple  vendors,  and  decreased  competition  efforts  to  get  competition  are  associated  with 
evidence  of  one-bid. 
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Regression  Model; 


Time  in  Weeks 


Effort  to  Compete 


One-bid 
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Table  10 

Discount  Regressions 

Dependent  Variable:  Discount  =  1  Yes,  =  0  No 


(1)  (2) 


Probit  Probit 


Hidden  Competition 


Log  (Contract  Value) 


Contract  Value 


(Contract  Value) 


Vendor  Overall 


Constant 


Difference  in  Predicted  probability  of  7.4%  6.5%  No  No 

discount,  time  in  weeks  change  from  0  to  2  significance  significance 

weeks  (percentage  points)^® 


These  regressions  were  estimated  using  the  PAE  data  set  described  in  Research  Methods. 
Standard  errors  are  given  in  parentheses  under  the  coefficients.  Individual  coefficients  are 
statistically  significant  at  the  *10%,  **5%  or  ***1%  level. 


Time  in  Weeks  is  positively  correlated  to  Discount.  The  predicted  probability  of  a  discoimt,  given  an 
increase  in  time  in  weeks  from  0  weeks  to  2  weeks  is  7.4%.  Increase  in  time  is  associated  with  increased 
probability  of  a  discount. 
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NONDISCLOSURE  STATEMENT 


I  acknowledge  that  my  conducting  a  policy  analysis  exercise  (PAE)  will  cause  me  to  have 
access  to  source  selection  or  proposal  information.  No  individual,  organization,  or 
contractor  will  be  identified  nor  any  product  or  process  associated  with  a  contractor  will 
be  identified  in  the  study  that  results  from  the  policy  analysis  exercise.  I  am  aware  that 
imauthorized  disclosure  of  source  selection  or  proposal  information  could  damage  the 
integrity  of  the  federal  procurement  system  and  that  the  transmission  of  relevant 
information  to  imauthorized  persons  could  subject  me  to  prosecution  under  the 
Procurement  Integrity  Laws  or  any  other  applicable  laws. 

I  will  not  divulge  copy,  publish,  or  reveal  by  word,  conduct,  or  any  other  means,  such 
information  or  knowledge.  Use  of  information  must  be  in  accordance  with  the  laws  of 
the  United  States,  unless  specifically  authorized  in  writing  in  each  and  every  case  by  a 
duly  authorized  representative  of  the  United  States  Government. 
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